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DoD  Acquisition  “Difficulties” 


41  “DoD’s  major  weapon  system  programs  continue  to  take  longer, 
cost  more,  and  deliver  fewer  quantities  and  (less)  capabilities 
than  originally  planned”  GAO 

The  DoD’s  Major  Defense  Acquisition  Projects  (MDAPs)  for 
2007  experienced 

-  An  average  delay  of  2 1  months 

-  An  average  program  cost  growth  of  26%,  representing 
approximately  $295  billion  dollars  in  additional  costs  (while  still 
reducing  quantities) 

4*  Cost  growth  is  a  primary  concern  as  DoD  plans  to  spend 

approximately  $935  billion  dollars  on  acquisition  between  fiscal 
years  2009  and  2013  (but  other  [non-discretionary]  pressures 
are  likely  to  reduce  the  acquisition  dollars  available). 
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Report,  years  covered 


Past  cost  growth  studies 


Arena,  2006 
McNicol,  2004 
Wandleland,  1993 
Drezner,  1993 
Tyson,  1989 
Asher,  1984 

0  0.5  1  1.5  2 

Cost  Growth  Factor  (CGF) 

□  Total  Program  CGF  ■  Procurement  CGF  □  Development  CGF 


Reported  Cost  Growth 


- 1 - 1 - 1 

1 

1 

^ "  1 

1  1  - 

? 

Center  for  Public  Policy  and  Private  Enterprise 


5/12/2010 

3 


Nunn-McCiirdv  Amendment 


Congress  tried  to  directly  address  the  issue  of 
unit  cost  growth  with  passage  of  the  Nunn- 
McCurdy  Amendment  (NM) 

-  Passed  in  1982,  made  permanent  in  1983 

+  NM  requires  DoD  to  report  when  unit  cost 
growth  of  any  major  defense  acquisition 
program  is  “known,  expected,  or  anticipated” 
by  a  program  manager  to  exceed  certain  cost 
growth  thresholds 
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NM  Breach  levels 


Two  levels  of  unit  cost  breach: 

^  A  significant  unit  cost  breach  occurred  if  a  program 
experienced  cost  growth  over  15%  of  the  current 
baseline  estimate 

#  A  critical  unit  cost  breach  occurred  if  a  program 
experiences  cost  growth  of  25%  over  the  current 
baseline  estimate. 

^  A  unit  cost  breach  occurs  if  a  program  experiences 
unit  cost  growth  above  specified  thresholds  as 
measured  by  either  program  acquisition  unit  cost 
(PAUC)  or  average  procurement  unit  cost  (APUC). 
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Reporting  requirements  bv  NM  breach  level 


^  For  a  significant  unit  cost  breach,  the  Service  Secretary  must 

-  notify  Congress  within  45  days  after  the  report  and 

-  submit  a  Selected  Acquisition  Report  (SAR)  with  required 
additional  unit  cost  breach  information 

41  For  a  critical  unit  costbreach,  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology  and  Logistics  must 
fulfill  all  'significant’  breach  requirements  and  must 
additionally  certify  to  Congress  within  60  days  of  the  SAR 
that  the  program  meets  four  criteria: 

1 .  the  system  is  essential  to  national  security; 

2.  there  are  no  alternatives  to  such  system  which  will  provide 
equal  or  greater  military  capability  at  less  cost; 

3.  the  new  estimates  of  the  unit  cost  are  reasonable;  and 

4.  the  management  stmcture  for  such  major  defense  system  is 
adequate  to  manage  and  control  unit  cost. 
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Impact  of  Original  NM 

Limited  data  restricts  ability  to  determine  impact  of 
NM  exclusively 

-  Overall,  program  acquisition  outcomes  have  not  improved 

significantly  since  implementation  of  NM 

41  Criticism  of  NM  as  not  effective  due  to  rebaselining 

-  Practice  of  establishing  a  new  “current”  baseline  to  avoid  a  NM  unit 
cost  breach 

-  Rebaselining  does  not  require  congressional  notification 

^  As  a  result,  few  programs  incurred  a  NM  breach 
between  1982-2004. 

-  Congress  has  recertified  the  majority  of  breached  programs 
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2006  revision  of  Niinn-McCiirdv 

+  The  NM  statute  was  amended  in  2006  to 
close  the  rebaselining  loophole. 

^New  provision:  unit  cost  growth  over  the 
original  baseline  estimate. 

-  A  significant  unit  cost  breach  occurs  when  cost  growth 
exceeds  30%  of  the  original  baseline 

-  A  critical  unit  cost  breach  occurs  when  cost  growth 
exceeds  50%  of  the  original  baseline  estimate. 

-  The  revision  did  not  change  the  reporting  requirements  for 
either  the  significant  or  critical  unit  cost  breach. 


Doubled  the  thresholds  and  included  “current”  as  well  as 

“original”  baseline 
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I  m nnc t  of  2006  revision 


DoD  reported  40  of  the  85  current  MDAP  programs  were 
experiencing  unit  cost  growth  high  enough  to  warrant  a  NM  breach. 

-  However,  25  programs  did  not  breach  because  the  National  Defense 
Authorization  Act  permitted  the  “original  baseline  estimate  to  be  revised  to  the 
current  baseline  estimate  as  of  January  6,  2006”  (Office  of  the  Under  Secretary 
of  Defense  (Acquisition  Resources  and  Analysis)  2006). 

-  Between  2006-2007,  16  additional  programs  experienced  a  NM  breach. 

Despite  increase  in  the  number  of  program  NM  breaches,  it  is  too 
soon  to  determine  the  long-term  impact  of  the  legislation  on  current 
acquisition  performance 

-  Immediate  short-term  impact  has  been  to  provide  greater  visibility,  as  well  as  a 
great  deal  more  emphasis  on  the  unit  cost  growth,  relative  to  the  original 
program  baseline. 
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The  Major  Weapons  Systems  Acquisition  Reform 
Act  of  2009 


^  Congress  again  amended  NM  with  the  Major  Weapons 
Systems  Acquisition  Reform  Act  of  2009.  This  law 
added  two  requirements  to  the  process  of  recertifying 
programs  that  incur  a  NM  breach. 

^  A  program  with  a  NM  unit  cost  breach  now  must 

-  (a)  rescind  the  most  recent  Milestone  approval  and 

-  (b)  receive  a  new  Milestone  approval  before  any  actions  regarding  the 
contract  may  continue.  The  new  Milestone  approval  requires  a 
certification  that  the  costs  of  the  program  are  reasonable,  and  the 
certification  must  be  supported  by  an  independent  cost  estimate  that 
includes  a  confidence  level  for  the  estimate. 

This  statute  was  implemented  too  recently  to  evaluate  its 
impact  upon  the  defense  acquisition  process. 
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Data  Analysis 


^  The  authors  conducted  a  data  analysis  using  the 

information  provided  by  the  Selected  Acquisition  Report 
Summary  Table  for  December  2007  (the  most  recent  full 
year  set  of  data  available  as  of  the  start  of  research). 

-  Analysis  of  one  SAR  summary  table  represents  a  snapshot  in  time. 


^  The  data  analysis  computed  several  tests  of 
independence  using  Fisher’s  “exact  test”. 

-  Fisher’s  “exact  test”  determines  the  “probability  of  getting  [an 
outcome]  as  strong  as  the  observed  or  stronger  simply  due  to  the 
chance  of  sampling”  (G.  David  Garson  2008). 

-  The  interpretation  of  the  Fisher’s  “exact  test”  p-value  O.Ox  is  that  there 
is  an  x%  chance  that,  given  the  information  provided  in  the 
contingency  table,  one  would  randomly  draw  an  outcome  as  strong  or 
stronger  than  the  sample  provided. 
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Data  Analysis  explanation 

Only  programs  in  development  3  years  or  longer  as  of 
the  December  2007  SAR  were  included  in  the  analysis, 
so  as  to  avoid  including  programs  too  new  to  have 
developed  significant  development  difficulties. 

-  The  data  set  contains  7 1  programs  that  meet  this  criteria 

Prior  SAR  summary  tables  were  consulted  to  determine 
if  a  program  has  incurred  a  NM  breach  or  high  unit  cost 
growth. 

Two  sets  of  programs  analyzed 

1 .  Programs  that  experienced  a  NM  breach  (18  programs) 

2.  Programs  that  experienced  a  NM  breach,  as  well  as  programs  that 
would  have  experienced  a  NM  breach  if  not  rebaselined  to  avoid  the 
new  NM  provisions  (3 1  programs) 
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Data  Analysis  examnle 

NM  Breach  by  Quantity  Change  for  FY  2007 


We  are  testing  to  see  if  NM  breach  is 
independent  of  quantity  change. 

Although  the  SAR  adjusts  the  cost  of 
the  program  for  changes  in  quantity, 
those  that  decrease  quantity  appear 
much  more  likely  to  breach. 

Interpretation:  Programs  that 
experience  a  NM  breach  are  either 
more  likely  to  reduce  quantity  in  order 
to  stay  within  total  program  cost 
thresholds  or  to  reduce  quantity  after 
incurring  a  breach. 


NM  Breach 
percentage 

No  Quantity 
Increase 

8% 

Quantity  Increase 

19% 

Quantity  Decrease 

48% 

Fisher’s  “exact  test”  estimates  there  is 
a  0.0%  chance  that,  if  quantity  change 
is  independent  of  breach,  that  this 
outcome  is  a  result  of  random  chance 
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Sample  Chart  of  SAR  Cost  Growth  “Causes 


Problem  Areas 

Specific  Problems 

Requirements 

Definition 

Poor  initial  requirements  definition 

Poor  performance/cost  trade-offs  during  development; 

Changes  in  quantity  requirements 

Cost  Estimating 

Errors  due  to  limitations  of  cost  estimating  procedures 

Failure  to  understand  and  account  for  technical  risks 

Poor  inflation  estimates 

Top  down  pressure  to  reduce  estimates 

Lack  of  valid  independent  cost  estimates 

Program 

Management 

Lack  of  program  management  expertise 

Mismanagement/human  error 

Over  optimism 

Schedule  concurrency 

Program  stretch  outs  to  keep  production  lines  open 

Technical 

Use  of  immature  technologies 

Adherence  to  strict  performance  requirements 

Reliance  on  proprietary  information 
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Samnle  Chart  of  SAR  Cost  Growth  “Causes” 


Contracting 

Lack  of  competition;  Contractor  buy-in  (to  win  competition) 

Use  of  wrong  type  of  contract 

Inconsistent  contract  management/administrative  procedures 

Too  much  contractor  oversight  and  reporting  requirements 

Waste 

Excess  profits 

Contractors  over  staffed 

Contractor  indirect  costs  unreasonable 

Taking  too  long  to  resolve  undefinitized  contracts 

Budgetary 

Funding  instabilities  within  DoD  caused  by  trying  to  fund  too  many 
programs 

Funding  instabilities  caused  by  congressional  decisions 

Inefficient  production  rates  due  to  stretching  out  programs 

Defense  Acquisition  Board  (DAB)— formerly  DSARC— out  of 
synchronization  with  the  services'  Program  Objective  Memorandum 
(POM)  cycle 

Failure  to  fund  for  management  reserve 

Failure  to  fund  programs  at  most  likely  cost 
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Data  analysis  conclusions 

This  data  analysis  produced  two  conclusions: 

First,  DoD’s  current  SAR  metrics  are  not  useful  for  determining  the 
root  cause  of  unit  cost  growth  in  acquisition  programs. 

41  Second,  despite  data  limitations,  it  appears  that  programs  that 

experience  high,  unit-cost  growth  are  not  randomly  distributed.  The 
two  most  important  factors  were  (1)  the  dollar  size  of  the  project  and 
(2)  the  “optimistic  estimating”  cost  category. 

41  Although  quantity  decrease  is  highly  correlated  with  NM  breach, 
quantity  decrease  is  more  likely  to  be  the  result  of  cost  growth 
(decreasing  units  to  stay  within  top  line  budgets)  as  opposed  to  the 
cause  of  cost  growth  (decreasing  quantity  because  the  mission 
changes)  -  particularly  as  the  SAR  unit  cost  growth  calculation 
adjusts  for  quantity. 
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Case  Study  #1 

4>  SBIRS-High: 

-  1996  contract  for  $4.5B  (FY2010  $)  with  launch  by  2004 
(covering  four  missions  “missile  warning,  missile  defense, 
technical  intelligence,  and  battlespace  characterization”  CRS) 

-  NM  breach  in  2001  (immature  technologies),  resulting  in  the 
program  being  restructured  and  rebaselined. 

-  Rebaselined  in  2005,  to  avoid  second  NM  breach 

-  Now  at  $13.5B  (FY2010  $)  with  launch  schedule  for  2011 

Conclusion:  NM  did  not  achieve  its  objective 
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Case  Study  #2 

*SSN  11 A 

-  1 993  contract  for  next  generation  of  attack  sub  (with  the 
objectives  of  lower  cost  and  greater  versatility) 

-  1998  contract  awarded  to  2  yard  “partnership”  for  $4.2B  (4 
ships) 

-  By  2003  program  cost  growth  24%,  so  Navy  initialed  cost 
reductions 

-  By  2008,  the  first  Block  II  ship  was  “delivered  eight  months 
ahead  of  schedule  and  $54  million  under  budget”  (Associated 
Press  2008) 

Conclusion:  Good  cost  management  by  DoD  can  be 
effective 
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Findings 

1 .  Unit  cost  growth  has  remained  high; 

2.  Few  programs  incurred  a  NM  breach  until  the  recent  2006  revision 
of  the  law  that  requires  programs  to  consider  unit  cost  growth  above 
a  program’s  original  baseline  (instead  of  the  current  authorized 
baseline); 

3.  DoD’s  data  collection  has  been  inconsistent  (regarding:  definitions; 
moving  baselines;  quantities;  etc.); 

4.  DoD  often  has  not  conducted  systematic  analysis  of  root-cause 
problems; 

5.  Limited  and  inconsistent  data  undermines  an  effective  analysis; 

6.  NM  may  identify  acquisition  problems  too  late  in  the  development 
process  to  allow  program  reforms  to  be  effective; 

7.  NM’s  effectiveness  may  be  limited  by  its  focus  on  the  development 
and  procurement  of  assets,  as  opposed  to  the  entire  life-cycle  of  the 
program;  and 

8.  Recent  legislation  has  not  been  implemented  long  enough  to 
evaluate  its  impact  on  DoD  acquisition  processes. 
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Recommendations 

41  In  order  to  control  cost  growth,  DoD  should 

1 .  Make  “unit  cost”  a  military  “requirement” 

2.  Implement  a  more  complete  acquisition  data  information  system 

3.  Consider  life-cycle  costs  when  rendering  acquisition  decisions 

4.  Directly  address  the  lack  of  incentives  (government  and 
budgetary)  that  allow  current  underlying  problems  to  persist 

5.  Work  with  Congress  to  increase  funding  flexibility  (e.g.  being 
able  to  use  production  and/or  O&M  money  to  increase 
development  costs  so  as  to  save  far  more  significant  unit 
production  and  support  costs) 

6.  Provide  programs  with  greater  requirements  flexibility  (e.g. 
allowing  cost/performance  trade-offs,  especially  for  “block  I”  of 
the  deployed  system,  so  that  the  last  5  to  1 0%  of  performance 
“requirements”  doesn’t  double  the  unit  costs) 
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Recommendations  (continued^ 

Regarding  NM,  DoD  should: 

1 .  Develop  a  system  to  determine  and  distribute  lessons- 
leamed  from  a  NM  breach,  throughout  the  DoD  and 

2.  Develop  leading  indicators. 

41  On  all  programs,  focus  on  using  internal  DoD 
management,  acquisition  strategy,  incentives,  and 
oversight  (by  P.M.,  service  acquisition  executives,  USD 
(AT&L),  etc.)  to  control  unit  costs  (and,  in  order  to  achieve 
the  required  performance,  quantity,  cost  and  schedule) 


This  can,  and  must,  be  done! 
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